Calibration device of camera, camera system, and calibration method of camera

ABSTRACT

To provide a calibration device of a camera, a camera system, and a calibration method of a camera which can achieve simplification of a calibration operation and improvement of accuracy of the calibration. A calibration device  2  includes a database  30 , an instruction section  25 , a correction section, and an evaluation section  24 . Process data in accordance with a specification of a camera  11 A is stored in the database  30 . The instruction section  25  outputs an operation instruction to a display screen  50 A based on the process data. The evaluation section  24  evaluates a correction image which is corrected by the correction section.

TECHNICAL FIELD

The present invention relates to a calibration device of a camera forobtaining camera parameters of a camera such as a monitoring camera or acamera sensor, a camera system having the calibration device, and acalibration method of a camera.

BACKGROUND ART

Conventionally, if distortion of an image of a monitoring camera, acamera sensor, and the like is corrected or in order to realize imagerecognition by a plurality of cameras such as stereo cameras, the cameraparameters, which are camera information indicating a focal length andorientation of the camera, and the like, are estimated.

It is conceivable that deviation occurs in the camera parameters whenmanufacturing the camera and when installing the camera on-site. Thecamera parameters are changed because distortion occurs in a housing dueto pressure in a contact surface when installing the camera or a housingmaterial of the camera is expanded or contracted by temperature,humidity, and the like. In addition, in the camera parameters, errorsalso occur over a long period of time by vibration and the like.Therefore, in the related art, calibration (correction) of the camera isperformed based on estimated camera parameters.

For example, a technique, which obtains data for calibration of a cameraby using a plurality of chart photographing images in which opticalconditions of the camera photographing a chart for calibration withmarks by changing the optical conditions, is described in PTL 1.

CITATION LIST Patent Literature

PTL 1: JP-A-2004-37270

SUMMARY OF INVENTION Technical Problem

However, if calibration is performed in a position in which the camerais installed, unlike factories manufacturing the camera, since equipmentfor performing the calibration is not always necessarily present in theposition in which the camera is installed, an operation may become verycomplicated.

In addition, the operation of calibration of a position in which areference chart is photographed, a size and the like is different forevery specification of a focal length of the installed camera, abaseline length, and the like. Therefore, an operator must select anoperation of calibration in accordance with the specifications of thecamera and the calibration operation is very complicated. As a result,variation occurs in the camera parameters estimated by the operator andthere is a problem that accuracy of the calibration decreases.

The invention is made in view of the problem described above and anobject of the invention is to provide a calibration device of a camera,a camera system, and a calibration method of a camera which can simplifya calibration operation and achieve improvement of accuracy of thecalibration.

Solution to Problem

In order to solve the problem described above and to achieve the objectof the invention, a calibration device of a camera of the inventionincludes a database, an instruction section, a correction section, andan evaluation section. The database stores process data that is anoperation step of calibration in accordance with a specification of thecamera. The instruction section outputs an operation instruction to adisplay screen based on the process data stored in the database. Thecorrection section corrects reference data obtained based on the processdata from image data photographed by the camera. The evaluation sectionevaluates a correction image corrected by the correction section.

In addition, a camera system of the invention includes a camera thatphotographs an image; and a calibration device that performs calibrationof the camera. For the calibration device, the calibration devicedescribed above is used.

Furthermore, a calibration method of a camera of the invention includesthe following steps indicated in from (1) to (4).

(1) A step of obtaining process data that is an operation step ofcalibration from a database in accordance with a specification of thecamera;

(2) A step of outputting an operation instruction to a display screen byan instruction section based on the obtained process data;

(3) A step of correcting reference data obtained based on the processdata from image data photographed by the camera by a correction section;and

(4) A step of evaluating a correction image corrected by the correctionsection by an evaluation section.

Advantageous Effects of Invention

According to the calibration device of a camera, the camera system, andthe calibration method of a camera of the invention, it is possible toachieve simplification of the calibration operation and improvement ofaccuracy of the calibration.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an entire configuration of acamera system according to an example of the invention.

FIG. 2 is an explanatory view illustrating an example of a table ofprocess data in a camera system according to a first example of theinvention.

FIG. 3 is a schematic view describing details of the process data in thecamera system according to the first example of the invention.

FIG. 4 is an explanatory view illustrating details of a photographingposition in the camera system according to the first example of theinvention.

FIGS. 5A and 5B are explanatory views illustrating details of thephotographing position in the camera system according to the firstexample of the invention.

FIG. 6 is a flowchart illustrating a calibration operation in the camerasystem according to the first example of the invention.

FIG. 7 is an explanatory view illustrating a display example of aninstruction that is output to a display screen by an instruction sectionin the camera system according to the first example of the invention.

FIG. 8 is an explanatory view illustrating a display example of aninstruction that is output to the display screen by the instructionsection in the camera system according to the first example of theinvention.

FIG. 9 is an explanatory view illustrating a display example of aninstruction that is output to a display screen by an instruction sectionin a camera system according to a second example of the invention.

FIG. 10 is an explanatory view illustrating an example of calibration inthe camera system according to the second example of the invention.

FIG. 11 is an explanatory view illustrating a display example of aninstruction output to a display screen by an instruction section in acamera system according to a third example of the invention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, examples of a calibration device of a camera, a camerasystem, and a calibration method of a camera of the invention will bedescribed with reference to FIGS. 1 to 11. Moreover, the same referencenumerals are given to common members in each view.

1. First Example 1-1. Configuration Example of Camera System

First, a configuration of a camera system according to a first example(hereinafter, referred to as the “example”) of the invention will bedescribed with reference to FIG. 1.

FIG. 1 is a block diagram illustrating an entire configuration of thecamera system of the example.

A camera system 1 illustrated in FIG. 1 is a stereo camera forphotographing a distance and a three-dimensional image using twocameras. As illustrated in FIG. 1, the camera system 1 has two cameras11A and 11B, a calibration device 2, an image recognition section 40,and a terminal device 50. The two cameras 11A and 11B, the calibrationdevice 2, the image recognition section 40, and the terminal device 50are connected via a wireless or wired network 7 (Local Area Network(LAN), Internet, dedicated line, and the like) and are able to transmitand receive data with each other.

ID numbers 11 a identifying specifications of cameras are respectivelyprovided in the two cameras 11A and 11B. Image data photographed by thetwo cameras 11A and 11B and the ID numbers 11 a are transmitted to thecalibration device 2, the image recognition section 40, and the terminaldevice 50 via the network 7.

The calibration device 2 has a calibration section 20 and a database 30in which various types of data are stored. The calibration section 20has an image obtaining section 21, a reference data storing section 22,a correction parameter calculation section 23, an evaluation section 24,an instruction section 25, and a process data retrieval section 26.

The image obtaining section 21 obtains the image data photographed bythe cameras 11A and 11B when performing calibration. Then, the imageobtaining section 21 extracts reference data that is used during acalibration operation from obtained image data and outputs the referencedata to the reference data storing section 22. In addition, the imageobtaining section 21 outputs the obtained image data to the evaluationsection 24.

The reference data output from the image obtaining section 21 is storedin the reference data storing section 22. Moreover, in the example, anexample in which the image obtaining section 21 extracts the referencedata from the image data, is described, but the invention is not limitedto the example. For example, the image data is output from the imageobtaining section 21 to the reference data storing section 22 and thereference data is extracted from the image data output by the referencedata storing section 22 and may be stored in the reference data storingsection 22. Furthermore, a reference data obtaining section extractingthe reference data from the image data obtained by the image obtainingsection 21 may be separately provided.

The reference data storing section 22 is connected to the correctionparameter calculation section 23. The correction parameter calculationsection 23 calculates correction parameters based on the reference datastored in the reference data storing section 22.

The correction parameter calculation section 23 outputs calculatedcorrection parameters to a correction parameter storing section 32 ofthe database 30 described below.

The image data obtained by the image obtaining section 21 and thecorrection parameters calculated by the correction parameter calculationsection 23 and stored in the correction parameter storing section 32 areoutput to the evaluation section 24. The evaluation section 24 generatesan evaluation correction image using the image data and the correctionparameters. In addition, evaluation data for evaluating the correctionimage is stored in the evaluation section 24. Then, the evaluationsection 24 evaluates a generated evaluation correction image based onthe evaluation data. The evaluation section 24 outputs information thatis evaluated to the instruction section 25.

Moreover, in the example, an example in which the evaluation section 24generates the evaluation correction image is described, but theinvention is not limited to the example. For example, a correctionsection generating the evaluation correction image based on thecorrection parameters and the image data may be provided in thecalibration section 20 or a correction image generated by a correctionsection 42 of the image recognition section 40 described below may beused.

The instruction section 25 receives information evaluated by theevaluation section 24 or the process data retrieved by the process dataretrieval section 26 described below. Then, the instruction section 25outputs the received information to the terminal device 50.

The process data retrieval section 26 is connected to a process datastoring section 31 and a camera installation data storing section 33 ofthe database 30 described below. The process data retrieval section 26retrieves and obtains the process data for performing the calibrationfrom the process data storing section 31 and the camera installationdata storing section 33 based on the ID numbers 11 a of the cameras 11Aand 11B. Then, the process data retrieval section 26 outputs theobtained process data to the instruction section 25.

Next, the database 30 will be described. The database 30 has the processdata storing section 31, the correction parameter storing section 32,and the camera installation data storing section 33.

Operation data for performing the operation of the calibration is storedin the process data storing section 31 for each specification of thecamera. In addition, camera installation data indicating a coordinateposition, a camera orientation, and the like for installing the cameras11A and 11B is stored in the camera installation data storing section33. Then, the process data that is an operation step for performing theoperation of the calibration is configured of the operation data and thecamera installation data.

FIG. 2 is an explanatory view illustrating an example of a table of theprocess data stored in the process data storing section 31 and thecamera installation data storing section 33. FIG. 3 is a schematic viewdescribing details of the process data.

As illustrated in FIG. 2, in the process data, a chart size 302, adistance 303, a photographing position 304, and a presence or absence ofshipping reference image 305 are set based on, for example, the IDnumber 11 a, a baseline length 300, and a focal length 301 of thecameras 11A and 11B.

As illustrated in FIG. 3, the baseline length 300 indicates a distancebetween lenses in the two cameras 11A and 11B. In addition, the focallength 301 is a distance from a principal point to the focal point ofthe lens of the camera 11A, that is, a distance from the lens to animaging element when the focus is adjusted. Thus, the focal length 301corresponds to a field angle 306 that is an angle of a range of a sceneto be photographed in an image photographed by the cameras 11A and 11Billustrated in FIG. 3. For example, as the focal length 301 becomesshorter, the field angle 306 becomes wider and as the focal length 301becomes longer, the field angle 306 becomes narrower.

The distance 303 in the process data indicates a distance between thecameras 11A and 11B, and a reference chart 201 photographed by thecameras 11A and 11B when performing the calibration.

In addition, the chart size 302 indicates a size of the reference chart201 photographed by the cameras 11A and 11B when performing thecalibration. The chart size 302 of a range, which fits in a common field308 of the two cameras 11A and 11B in the distance 303, is set if thefield angle 306 and the baseline length 300 of the cameras 11A and 11Bare determined.

Next, details of the photographing position 304 indicated in the processdata will be described with reference to FIGS. 4 and 5.

FIGS. 4 and 5 are explanatory views describing the details of thephotographing position.

The photographing position 304, for example, as illustrated in FIG. 4,fixes the reference chart 201, moves the camera 11A, and sets a positionin which the reference data is obtained. In the example illustrated inFIG. 4, an example, in which the reference chart 201 is photographedfrom four directions of four points Q1, Q2, Q3, and Q4, is illustrated.

In addition, the photographing positions 304 are appropriately set bycorrection accuracy required by the calibration and are preferably setto be equal to or greater than five positions of four upper, lower,right, and left positions, and a front with respect to the referencechart 201.

Moreover, in the example illustrated in FIG. 4, an example in which thecamera 11A is moved is described, but the invention is not limited tothe example. For example, if the camera 11A is fixed, as illustrated inFIGS. 5A and 5B, the reference chart 201 may be moved with respect tothe camera 11A. In this case, an operation is performed by setting thedistance between the camera 11A and the reference chart 201 as adistance that is set by the distance 303.

The presence or absence of shipping reference image 305 sets whether ornot an image (hereinafter, referred to as “shipping reference image”) inwhich the reference chart 201 is photographed under the same conditionsas those during the operation of the calibration is present whenshipping the cameras 11A and 11B. If the shipping reference imageindicating an example of an initial reference image is present, it ispossible to estimate displacement of camera parameters by comparing theshipping reference image to the image data photographed during thecalibration operation, so-called reference data.

In addition, when the evaluation section 24 evaluates the correctionparameters, the evaluation may be performed using the shipping referenceimage.

Then, for a plurality of types of process data for each specification ofthe camera described above, the process data is retrieved in accordancewith a camera in which the calibration is performed by the process dataretrieval section 26 of the calibration section 20. Then, the retrievedprocess data is output to the instruction section 25 via the processdata retrieval section 26.

The correction parameters calculated by the correction parametercalculation section 23 are stored in the correction parameter storingsection 32. The correction parameter storing section 32 outputs thestored correction parameters to the evaluation section 24 or thecorrection section 42 of the image recognition section 40 describedbelow. In addition, the correction image that is corrected by the imagerecognition section 40 described above may be output to the correctionparameter storing section 32.

The image recognition section 40 has an image obtaining section 41, thecorrection section 42, and an image processing section 43. The imageobtaining section 41 obtains the image data from the two cameras 11A and11B. The correction section 42 corrects the image data obtained by theimage obtaining section 41 based on the correction parameters outputfrom the correction parameter storing section 32 and generates thecorrection image. The correction section 42 outputs the generatedcorrection image to the correction parameter storing section 32 or theimage processing section 43.

The image processing section 43 processes the correction image outputfrom the correction section 42 and calculates a parallax of right andleft of the two cameras 11A and 11B. Moreover, in the example, theprocess of the stereo camera is described in the image processingsection 43, but the invention is not limited to the example. The imageprocessing section 43 may process, for example, an image for thethree-dimensional image from the correction image output from thecorrection section 42.

The terminal device 50 is a display device having, for example, adisplay screen 50A formed of a liquid crystal display screen, a CathodeRay Tube (CRT) display screen, and the like and an input section. Imagesphotographed by the cameras 11A and 11B, and instruction informationoutput from the instruction section 25 are displayed in the displayscreen 50A. Moreover, details of the instruction information displayedin the display screen 50A of the terminal device 50 will be describedlater.

Moreover, in the camera system 1 of the example, the two cameras 11A and11B and the calibration section 20, the database 30, the imagerecognition section 40, and the terminal device 50 are connected via thenetwork 7, and are respectively described as separate configurations,but the invention is not limited to the example. For example, thecalibration section 20, the database 30, the image recognition section40, and the terminal device 50 may be built into the camera 11A.

In addition, instead of providing the terminal device 50, aconfiguration, in which a Red Green Blue monitor output and a dataoutput via a network are performed to a liquid crystal display deviceand a CRT display device of an external apparatus such as a personalcomputer (PC), may be provided.

1-2. Calibration Method of Camera

Next, a calibration method of the cameras 11A and 11B in the camerasystem 1 of the example will be described with reference to FIGS. 6 to8.

FIG. 6 is a flowchart illustrating the calibration operation.

First, as illustrated in FIG. 6, the calibration section 20 obtains theprocess data for performing the calibration operation from the database30 (step S1). Specifically, first, the calibration section 20 obtainsthe ID numbers 11 a identifying the specification of the camera from thecameras 11A and 11B. Then, the process data retrieval section 26retrieves the process data from the process data storing section 31 andthe camera installation data storing section 33 based on the obtained IDnumber 11 a and obtains the process data.

Next, the process data retrieval section 26 outputs the obtained processdata to the instruction section 25. Then, the instruction section 25outputs the operation instruction to the display screen 50A of theterminal device 50 based on the output process data (step S2). Forexample, the instruction section 25 outputs the table of the processdata illustrated in FIG. 2 to the display screen 50A of the terminaldevice 50.

Thus, an operator can easily determine operation contents for performingthe calibration and achieve simplification of the calibration operation.

In addition, the instruction section 25 may output the operationinstruction to the terminal device 50 based on the image data obtainedin step S3 described below.

Here, an example of instruction contents displayed in the display screen50A will be described with reference to FIGS. 7 and 8.

FIGS. 7 and 8 are views illustrating a display example of theinstruction output to the display screen 50A by the instruction section.

As illustrated in FIG. 7, a first display section 51A in which an imagephotographed by the camera 11A is displayed and a second display section51B in which an image photographed by the camera 11B is displayed areprovided in the display screen 50A. Moreover, the terminal device 50 hasan image obtaining section for obtaining the image data photographed bythe cameras 11A and 11B. A first guide 52A is displayed in the firstdisplay section 51A and a second guide 52B is displayed in the seconddisplay section 51B. The first guide 52A and the second guide 52B areappropriately changed based on the process data.

Furthermore, an operation instruction window 53 displaying the operationcontents with respect to the operator is displayed in the display screen50A.

The operator moves the reference chart 201 or the cameras 11A and 11B sothat frames of the first guide 52A and the second guide 52B displayed inthe display screen 50A and an outer edge of the reference chart 201 arefitted with each other. In addition, the instruction section 25 outputsthe operation instruction to the operation instruction window 53 inaccordance with the positions of the reference chart 201 displayed inthe first display section 51A and the second display section 51B.

The instruction section 25 measures distances between four corners ofthe reference charts 201 displayed in the first display section 51A andthe second display section 51B and four corners of the first guide 52Aand the second guide 52B. Then, the instruction section 25 outputs theoperation instruction to the operation instruction window 53 inaccordance with a measured distance. Moreover, comparison between thereference charts 201 displayed in the first display section 51A and thesecond display section 51B, and the first guide 52A and the second guide52B may be determined so as to minimize errors of lengths of four sides.

Otherwise, as illustrated in FIG. 8, the instruction section 25calculates positions in a real space in the cameras 11A and 11B, and thereference chart 201 based on the obtained image data, and may output thepositions to the display screen 50A. Then, the instruction section 25displays an arrow 54, a guide 55, and the like on the display screen 50Aand may instruct the operator to position the reference chart 201 in apredetermined position.

As described above, the instruction section 25 instructs an operationprocedure in the calibration operation so as to be capable of accuratelytransmitting the operation contents to the operator and it is possibleto prevent variation of accuracy of the calibration operation accordingto the operator. Therefore, it is possible to improve the accuracy ofthe calibration.

Next, the calibration section 20 obtains the image data photographed bythe cameras 11A and 11B using the image obtaining section 21 (step S3).Then, the image obtaining section 21 obtains the reference datanecessary for the calibration among a plurality of types of obtainedimage data (step S4). Moreover, the reference data is the image datathat is fitted with conditions of the process data among the pluralityof types of image data. Then, the image obtaining section 21 outputs theobtained reference data to the reference data storing section 22.

Next, the correction parameter calculation section 23 calculates thecorrection parameters based on the reference data stored in thereference data storing section 22 (step S5).

Specifically, first, the correction parameter calculation section 23estimates a camera parameter C formed of an internal parameter K and anexternal parameter D of the cameras 11A and 11B based on the referencedata. Then, the correction parameter calculation section 23 calculates adeviation amount between an estimated camera parameter C and thereference parameter that is set in advance. Next, the correctionparameter calculation section 23 calculates the correction parameters sothat the deviation amount becomes the minimum value or 0. Moreover, thecalculation of the correction parameters is calculated by repeating thecalculation process a predetermined number of times.

Here, the camera parameter C will be described.

First, the internal parameter K is represented by a matrix illustratedin the following Equation 1. In Equation 1, f indicates a focal length,a indicates an aspect ratio, s indicates skew indicating distortion ofan image, and (v_(c) and u_(c)) indicate center coordinates ofcoordinates (image coordinates) within images photographed by thecameras 11A and 11B.

$\begin{matrix}{K = \begin{bmatrix}f & {sf} & v_{c} & 0 \\0 & {af} & u_{c} & 0 \\0 & 0 & 1 & 0\end{bmatrix}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In addition, the external parameter D is represented by a matrixillustrated in the following Equation 2. In Equation 2, (r₁₁, r₁₂, r₁₃,r₂₁, r₂₂, r₂₃, r₃₁, r₃₂, and r₃₃) indicate the orientations of thecameras 11A and 11B. In addition, in Equation 2, (t_(X), t_(Y), andt_(Z)) indicate coordinates of the positions in which the cameras 11Aand 11B are installed, so-called world coordinates.

$\begin{matrix}{D = \begin{bmatrix}r_{11} & r_{12} & r_{13} & t_{X} \\r_{21} & r_{22} & r_{23} & t_{Y} \\r_{31} & r_{32} & r_{33} & t_{Z} \\0 & 0 & 0 & 1\end{bmatrix}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Then, image coordinates (u, v) and the world coordinates (X_(W), Y_(W),Z_(W)) can be represented by a relational expression illustrated in thefollowing Equation 3 using the internal parameter K and the externalparameter D. Moreover, in Equation 3, λ indicates a constant.

$\begin{matrix}{{\lambda \begin{bmatrix}u \\v \\1\end{bmatrix}} = {{KD}\begin{bmatrix}X_{w} \\Y_{w} \\Z_{w} \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

Moreover, when (r₁₁, r₁₂, r₁₃, r₂₁, r₂₂, r₂₃, r₃₁, r₃₂, and r₃₃)representing the orientations of the cameras 11A and 11B in the externalparameter D are defined by Euler angles, they are represented by threeparameters of a pan θ, a tilt φ, and a roll ψ that are installationangles of the cameras 11A and 11B. Thus, the number of camera parametersnecessary for associating the image coordinates and the worldcoordinates becomes eleven that is a sum of five internal parameters andsix external parameters.

In addition, it can be seen that the matrix of the camera parameter C inEquation 3 is unchanged in its meaning even if the matrix is constantmultiplication. Thus, the constant λ, the internal parameter K, and theexternal parameter D are organized in one matrix and can be representedas illustrated in the following Equation 4.

$\begin{matrix}{\begin{bmatrix}u \\v \\1\end{bmatrix} = {\begin{bmatrix}c_{11} & c_{12} & c_{13} & c_{14} \\c_{21} & c_{22} & c_{23} & c_{24} \\c_{31} & c_{32} & c_{33} & c_{34}\end{bmatrix}\begin{bmatrix}X_{w} \\Y_{w} \\Z_{w} \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Furthermore, in Equation 4, when c₃₄=1, the number of unknown parametersis eleven. Then, determining values of eleven parameters is synonymouswith determining five internal parameters and six external parameters inEquation 3. Therefore, Equation 3 can be converted into the followingEquation 5. Then, a matrix C in Equation 5 becomes the camera parameterC.

$\begin{matrix}\begin{matrix}{\begin{bmatrix}u \\v \\1\end{bmatrix} = {C\begin{bmatrix}X_{w} \\Y_{w} \\Z_{w} \\1\end{bmatrix}}} & \left( {C = \begin{bmatrix}c_{11} & c_{12} & c_{13} & c_{14} \\c_{21} & c_{22} & c_{23} & c_{24} \\c_{31} & c_{32} & c_{33} & c_{34}\end{bmatrix}} \right)\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

Moreover, the correction parameter calculated by the correctionparameter calculation section 23 may be the camera parameter Cillustrated in Equation 5 or may be only the internal parameter Killustrated in Equation 1.

Next, the correction parameter calculation section 23 stores thecalculated correction parameter in the correction parameter storingsection 32. Then, the evaluation section 24 or the correction section 42corrects the image data based on the stored correction parameter (stepS6). In the correction process in step S6, a process is selected inaccordance with application of the camera system 1.

Next, the evaluation section 24 evaluates the evaluation correctionimage based on the evaluation data that is stored in advance. That is,the evaluation section 24 performs determination on whether or not thecorrection image is within a range of the evaluation data (step S7). Asan evaluation method performed by the evaluation section 24, forexample, distance data calculated from the correction image and anactual measurement value are compared or deviation between the shippingreference image and the correction image are evaluated in a case inwhich the shipping reference image is present.

Otherwise, the evaluation section 24 detects the number E of edge pointsin an evaluation table correction image and calculates a parallaxdensity D from an effective number d of parallaxes. The parallax densityD is calculated from the following Expression 1.

D=d/E  [Expression 1]

Expression 1 represents the effective number of parallaxes with respectto the number of edge points and may be used as an evaluation value forthe accuracy of the correction parameter and for whether it is parallelin stereo processing. Then, the evaluation section 24 may performevaluation of the correction image by determining whether or not thecalculated parallax density D is within a range of a threshold value.

In the process of step S7, if the evaluation section 24 determines thatthe correction image is within the range of the evaluation data (YESdetermination of step S7), the evaluation section 24 outputs anevaluation result to the instruction section 25. Then, the calibrationsection 20 stores the evaluated correction parameter in the correctionparameter storing section 32. Therefore, the calibration operation iscompleted.

In addition, in the process of step S7, if the evaluation section 24determines that the correction image is not within the range of theevaluation data (NO determination of step S7), the evaluation section 24outputs the evaluation result to the instruction section 25. Then, thecalibration section 20 performs the calibration operation again byreturning to the process of step S2.

As described above, in the camera system 1 of the example, it ispossible to determine the correction parameter by the evaluation section24 by evaluating the calculated correction parameter by the evaluationsection 24 even if the accuracy of the calibration is reduced. Then,information evaluated by the evaluation section 24 is output via theinstruction section 25. As a result, the operator can quickly recognizethat the accuracy of the calibration is reduced and improve the accuracyof the calibration.

In addition, if the calibration operation is incorrect, it can bedetected by the evaluation section 24.

Moreover, in the example, an example in which the calibration section 20obtains the ID numbers 11 a provided in the cameras 11A and 11B isdescribed, but the invention is not limited to the example. For example,when performing the calibration operation, the operator inputsidentification information of the camera such as the ID numbers of thecameras 11A and 11B, the baseline length, and the focal length into theterminal device 50. Then, the terminal device 50 may transmit the inputidentification information of the camera to the calibration device 2.

2. Second Example

A camera system according to a second example of the invention will bedescribed with reference to FIG. 9.

FIG. 9 is an explanatory view illustrating instruction contents of aninstruction section in the camera system according to the secondexample.

A difference point of the camera system according to the second examplefrom the camera system 1 according to the first example, is theinstruction contents being output to the terminal device 50 by theinstruction section 25. Thus, here, contents that are displayed in adisplay screen 50A of the terminal device 50 will be described and thesame reference numerals are given to common portions to the camerasystem 1 according to the first example and redundant description willbe omitted.

Here, in a case in which a distance to the cameras 11A and 11B withrespect to a certain object is known in advance, it is possible toobtain an expected value of the parallax of the two cameras 11A and 11B.For example, in a case in which the object is a plane normally facingthe cameras 11A and 11B, the parallax is constant. Furthermore, in acase in which the object is disposed to be inclined with respect to thecameras 11A and 11B, it is possible to optimize a result of thecalculated parallax by detecting angles of the object and the cameras11A and 11B.

FIG. 9 is an explanatory view illustrating display contents displayed inthe display screen 50A.

As illustrated in FIG. 9, images that are photographed by the twocameras 11A and 11B are displayed in the display screen 50A. Inaddition, a distance input window 56 into which an actually measuredvalue from the cameras 11A and 11B to a selected object is input isdisplayed in the display screen 50A.

The instruction section 25 instructs the operator to select so-calledplanes 500, 501, and 502 of the object which are known in advance via acursor 50B, in which the distance to the cameras 11A and 11B is known.In addition, the instruction section 25 instructs the operator to inputthe distance from the cameras 11A and 11B to the selected planes 500,501, and 502 into the distance input window 56.

Here, a parallax d with respect to a distance Z can be calculated fromthe following Expression 2. Moreover, B indicates the baseline lengthand f indicates the focal length.

D=B×f/Z  [Expression 2]

Then, the calibration section 20 performs correction so that theparallax becomes close to the expected value by changing the values ofthe internal parameter K illustrated in Equation 1 and the cameraparameter C illustrated in Equation 5 based on the region and thedistance that are input. In addition, the calibration of the parallax isperformed by using a bundle adjustment method used in a survey field,that is, by minimizing a square sum of an error.

FIG. 10 is an explanatory view describing an example of the calibration.

In addition, the invention is not limited to the plane and a correctionvalue of the calibration may be obtained by fitting of a straight lineand the like so that linear data 1001 obtained in a floor surface, aceiling surface, and the like becomes ideal linear data 1000.

3. Third Example

Next, a camera system according to a third example of the invention willbe described with reference to FIG. 11.

FIG. 11 is an explanatory view illustrating instruction contents of aninstruction section in the camera system according to the third example.

A difference point of the camera system according to the third examplefrom the camera system 1 according to the first example, is theinstruction contents being output to the terminal device 50 by theinstruction section 25 and a range of an evaluation data of theevaluation section 24. Thus, here, contents that are displayed in thedisplay screen 50A of the terminal device 50 will be described and thesame reference numerals are given to common portions to the camerasystem 1 according to the first example and redundant description willbe omitted.

As illustrated in FIG. 11, an image photographed by the two cameras 11Aand 11B is displayed in the display screen 50A. Then, the image isdivided into nine regions 401A, 401B, 401C, 401D, 401E, 401F, 401G,401H, and 401I.

Then, a value of the evaluation data of the evaluation section 24 ischanged for each of the regions 401A, 401B, 401C, 401D, 401E, 401F,401G, 401H, and 401I. For example, a region through which importantpeople and objects pass is set so that the range of the evaluation datathereof is strict compared to that of other regions.

In addition, the instruction section 25 may prompt the operator toselect a region in which an image is evaluated via a cursor 50B.

Therefore, it is also possible to improve performance of only a specificregion in the image photographed by the two cameras 11A and 11B.

Moreover, the region to be divided is not limited to nine regions andthe image is divided into equal to or less than eight regions or equalto or greater than ten regions, and the region may be set so that theevaluation data of the evaluation section 24 is different.

Moreover, the invention is not limited to the examples described aboveand illustrated in the drawings, and various modifications can be madewithout departing from the gist of the invention described in theclaims.

In the examples described above, an example, in which the stereo camerahaving two cameras is applied as the camera system, is described, butthe invention is not limited to the example. The camera system can beapplied to, for example, a camera system having equal to or greater thanthree cameras, a monitoring camera formed of one camera, and the like.Furthermore, the camera system can be applied to various other camerasystems such as a three-dimensional video camera photographing athree-dimensional image, a vehicle-mounted camera, an objectiveinterpersonal sensing camera for sensing the people and the objects, anautomatic driving system, a people flow measurement system, and a videosecurity system.

REFERENCE SIGNS LIST

1 . . . camera system, 2 . . . calibration device, 7 . . . network, 11 a. . . ID number, 11A, 11B . . . camera, 20 . . . calibration section, 21. . . image obtaining section, 22 . . . reference data storing section,23 . . . correction parameter calculation section, 24 . . . evaluationsection, 25 . . . instruction section, 26 . . . process data retrievalsection, 30 . . . database, 31 . . . process data storing section, 32 .. . correction parameter storing section, 33 . . . camera installationdata storing section, 40 . . . image recognition section, 41 . . . imageobtaining section, 42 . . . correction section, 43 . . . imageprocessing section, 50 . . . terminal device, 50A . . . display screen,53 . . . operation instruction window, 201 . . . reference chart, 300 .. . baseline length, 301 . . . focal length, 306 . . . field angle

1. A calibration device of a camera comprising: a database in which process data that is an operation step of calibration in accordance with a specification of the camera is stored; an instruction section that outputs an operation instruction to a display screen based on the process data stored in the database; a correction section that corrects reference data obtained based on the process data from image data photographed by the camera; and an evaluation section that evaluates a correction image corrected by the correction section.
 2. The calibration device of a camera according to claim 1, wherein the instruction section outputs an evaluation result evaluated by the evaluation section to the display screen and instructs a calibration operation again in accordance with the evaluation result.
 3. The calibration device of a camera according to claim 1, further comprising: a process data retrieval section that retrieves and obtains the process data in accordance with the specification of the camera from the process data stored in the database.
 4. The calibration device of a camera according to claim 3, wherein the process data retrieval section retrieves and obtains the process data in accordance with the specification of the camera from the process data stored in the database by obtaining an ID number that identifies the specification of the camera from the camera.
 5. The calibration device of a camera according to claim 1, wherein the instruction section outputs a guide so as to display the guide for guiding a position of a reference chart photographed by the camera on the display screen based on the process data.
 6. The calibration device of a camera according to claim 1, wherein an initial reference image photographed during initial setting of the camera is stored in the process data, and wherein the evaluation section evaluates the correction image using the initial reference image.
 7. The calibration device of a camera according to claim 1, wherein the correction section generates the correction image by selecting a plane of which a distance from an image that is photographed by the camera to an object of the camera is known and inputting the distance.
 8. A camera system comprising: a camera that photographs an image; and a calibration device that performs calibration of the camera, wherein the calibration device includes a database in which process data that is an operation step of calibration in accordance with a specification of the camera is stored, an instruction section that outputs an operation instruction to a display screen based on the process data stored in the database, a correction section that corrects reference data obtained based on the process data from image data photographed by the camera, and an evaluation section that evaluates a correction image corrected by the correction section.
 9. A calibration method of a camera comprising: a step of obtaining process data that is an operation step of calibration from a database in accordance with a specification of the camera; a step of outputting an operation instruction to a display screen by an instruction section based on the obtained process data; a step of correcting reference data obtained based on the process data from image data photographed by the camera by a correction section; and a step of evaluating a correction image corrected by the correction section by an evaluation section. 